A Framework of Learning Through Empirical Gain Maximization

We develop in this letter a framework of empirical gain maximization (EGM) to address the robust regression problem where heavy-tailed noise or outliers may be present response variable. The idea EGM is approximate density function distribution instead approximating truth directly as usual. Unlike classical maximum likelihood estimation that encourages equal importance all observations and could problematic presence abnormal observations, schemes can interpreted from minimum distance viewpoint allow ignorance those observations. Furthermore, we show several well-known nonconvex paradigms, such Tukey truncated least square regression, reformulated into new framework. then learning theory for by means which unified analysis conducted these well-established but not fully understood approaches. This leads novel interpretation existing bounded loss functions. Within framework, two seemingly irrelevant terminologies, Tukey's biweight triweight kernel nonparametric smoothing, are closely related. More precisely, derived kernel. Other frequently employed functions machine learning, loss, Geman-McClure exponential squared also certain smoothing kernels statistics. In addition, enables us devise learning.